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ABSTRACT 


Parallel to ICT investments in schools, monitoring programs have been set up to gain insight into the “return on 
investment”, to assess their effects on educational practices and to map trends. This paper presents the theoretical and 
methodological framework of a region-wide ICT monitoring study (MICTIVO) conducted in Dutch-speaking schools in 
Belgium. In line with the MICTIVO model, ICT integration is conceived as a unity consisting of ICT infrastructure, ICT 
policy and ICT use at the micro-level, in which three actors are involved: principals, teachers and pupils. First, this article 
focusses on the two European monitoring studies that have inspired MICTIVO: Four in Balance Monitor and Eurydice. 
Second, this article discusses each of the MICTIVO components in detail. Thirdly, two elements of the field study are 
highlighted: the sample design and the data collection. The scales were empirically validated in three large-scale 
MICTIVO studies and show strong psychometric properties. Due to its theoretically-grounded approach and its 
methodological strengths we believe MICTIVO allows researchers and practitioners to study ICT integration in a 
comprehensive, representative way. 


KEYWORDS 
ICT Monitor, Schools, MICTIVO, ICT Integration 


1. INTRODUCTION 


In Belgium, two regional governments are responsible for educational policy. With regard to the 
Dutch-speaking schools in Belgium, the Flemish government has been investing in the diffusion of 
information and communication technology (ICT) for over 20 years (Evers, Sinnaeve, Clarebout, van Braak, 
& Elen, 2009). The types of investments are diverse: equipment and applications, project funding, 
professional development programs and coordinating activities and expert staff. In 2007, cross-curricular 
objectives concerning the integration of ICT were introduced in Flemish education in the form of attainment 
targets (Vandenbroucke, 2007). Ever since, schools subsidized by the government are expected to integrate 
ICT in education. Schools autonomously decide how these objectives will be reached, and how ICT are 
employed to support and enhance teaching and learning. Over the years several important questions 
regarding the effectiveness of such government policy have come to the forefront. These are related to one 
particular point of interest: "To what extent do schools implement and integrate ICT in their education?". 
To answer this question, a monitor for ICT integration in Flemish education (MICTIVO) was developed. 

The goal of MICTIVO is to take a representative snapshot of ICT integration in Flemish education. 
The main research objectives are: 

1. Ongoing development and validation of a monitoring instrument based on important 
evolutions in ICT and education, and based on a number of (new) policy priorities; 

2. A large-scale and representative measurement of ICT integration in primary, secondary and 
basic education on the basis of a web survey, and the reporting of results at the system 
level; 

3. Comparative analyses based on 5-year results gathered since its first round in 2007-2008. 
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The starting point for MICTIVO is a research-based model of factors that relate to ICT integration (Evers, 
Sinnaeve, Clarebout, van Braak, & Elen, 2009). This model includes four components: infrastructure and 
policy, perceptions, competences and usage at the micro level. These components relate to three actors: 
principals, teachers and pupils with all of these actors having their particular background, ICT competences 
and perceptions on ICT (see Figure 1). The core components of the original model from 2007 are maintained 
to facilitate a comparison at different measurement points in time. Evenly important to note is that MICTITVO 
was conceived as a recurrent monitor with room to add indicators that measure latest [CT-related innovations 
(e.g. social media usage, BYOD). This paper focuses on the methodological strength of MICTIVO, in 
comparison with other monitors. 


ICT- 
Policy makers 
(characteristics, 
competences, 
perceptions) 


(Supportive) context 
ICT-integration 


Educational learning 
process (ICT-use at 
micro level) 


Students 
(characteristics, 
competences. 
perceptions) 


Teachers 
(characteristics, 
competences, 
perceptions) 


Figure 1. MICTIVO model (Evers, Sinnaeve, Clarebout, van Braak, & Elen, 2009) 


2. RESEARCH FRAMEWORK 


2.1 ICT Integration Monitoring in Europe 


In 2017, a report on Digital Education Policies in Europe and Beyond (Conrads, Rasmussen, Winters, Geniet, 
Langer, & 2017) was published stating that the integration and innovative use of digital technologies in 
education has become a policy priority across Europe. The report further emphasizes that careful monitoring 
and evaluation are crucial for supporting innovation. 

In- and outside the European Union, a couple of monitoring instruments are employed for mapping the 
adoption and diffusion of technology for teaching and learning purposes. The European Schoolnet and the 
University of Liege (2013), for example, carried out a study on ICT in education in 27 European countries, 
including Belgium. Data was collected from school leaders, teachers and pupils about elements such as ICT 
infrastructure, ICT use, ICT competences and attitudes (e.g. Fraillon, Ainley, Schulz, Friedman, & Gebhardt, 
2014; European Commission, 2017; Wagner et al., 2005; Collie, Lewis, & Méro, 2011). These efforts 
provide opportunities for policy preparation, evaluation, quality improvement and benchmarking (Goeman, 
Elen, Pynoo, & van Braak, 2013). 

MICTIVO was developed in 2007 to assess the impact of ICT at all levels of formal education in the 
Dutch part of Belgium. In 2012-2013 a follow-up study took place (MICTIVO2), while in September 2017 
the third and present study has started (MICTIVO3). MICTIVO is originally inspired by two European 
monitoring studies, namely The four in Balance monitor and Eurydice. The Four in Balance Monitor 
(Stichting Kennisnet ICT op school, 2006; Kennisnet, 2017) is published annually by Kennisnet and pertains 
to the use and benefits of ICT in schools in the Netherlands. This model stipulates that introducing ICT for 
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educational purposes has a greater chance of success if the following elements are in balance: vision, 
expertise, digital learning materials, and ICT infrastructure. To balance these elements, collaboration and 
leadership is required. The balance that will be obtained, provides opportunities of support for using ICT in 
teaching and learning. Eurydice is a network that provides reports on European education systems and 
policies and thereby provides insights into ICT-related developments in different countries. The report ‘Key 
Data on Learning and Innovation through ICT at School in Europe 2011’ (Ranguelov, Horvath, Dalferth, 
& Noorani, 2011) builds on the previous Eurydice publications on ICT in schools in Europe. It provides an 
overview of how ICT infrastructure has evolved, how ICT is used in educational processes and how ICT is 
included in European curricula. Data was gathered in primary and secondary education of 31 European 
countries. 


2.2 The Components of MICTIVO3 


One of the main purposes of MICTIVO is to get a comprehensive view on ICT integration in education. 
In MICTIVO1 an extensive literature study was conducted to create a model with four different components 
and three different actors (Evers, Sinnaeve, Clarebout, van Braak, & Elen, 2009). 

Each of the components consists of different indicators that were validated in MICTIVO1 (2007-2008) 
using exploratory and confirmatory factor analysis. After each field study reliability measures were 
calculated in order to assess the internal consistency of the scales. In MICTIVO2 (2012-2013) new indicators 
were added: media literacy (defined in accordance to Lieten & Smet, 2012), the use of social media and 
digital games for educational purposes and professional development (adapted from Vanderlinde & van 
Braak, 2010). Logically, additional factor analyses were conducted for the new scales. In the latest 
MICTIVO study no new indicators were added nor were existing scales adjusted. Hence, new factor analyses 
were not needed. 

In the table below (Table 1) all components and their related indicators are given, as well as the actors 
that were questioned for that indicator. For the component on ICT policy and infrastructure, the indicators of 
available hardware (e.g. number of devices, origin, age), software (e.g. website, electronic learning 
environment) and policy (e.g. origin of policy, support for teachers, security) were measured. The second 
component, ICT integration, assesses the use of ICT by teachers and pupils in- and outside of the classroom 
context (e.g. to prepare classes, to support classroom activities, for homework purposes). The ICT 
competences component consists of the indicators concerning pedagogical-didactical competences of 
teachers (e.g. class management, evaluation), competences of pupils (e.g. ability of making a presentation, 
finding relevant information on the internet) and computer experience (e.g. years of computer experience, 
amount of time spent on an ICT device for leisure). The fourth component, ICT perceptions, consists of the 
following indicators: perceptions on the importance of ICT use in and for education (e.g. importance from an 
educational and economical rational), digital literacy (e.g. attitudes and knowledge on responsible use of 
ICT), use of social media (e.g. using social media for communication with pupils), use of educational games 
(e.g. supporting pupils with a disability through educational games) and professional development 
(e.g. amount of trainings on ICT use in education). 


Table 1. Components and indicators of MICTVO3 for the different actors 
Component 


indicators 


ICT policy and infrastructure 
availability of hardware 


availability of software 
ICT policy 
professionalization 


ICT integration 
ICT use by teachers 
ICT use by pupils 
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ICT competences 


pedagogical-didactical competences x : 
competences of pupils x x x 
computer experience Xx Xx 
media literacy x x x 


ICT perceptions regarding 
importance of ICT for education 


infrastructure 
professional development 


2.3 Sample Design and Participants 


In order to achieve the research objectives, the sample design is focused on getting a representative sample of 
schools participating to the field study. Since its first round in 2007, MICTIVO includes one fifth of all 
Flemish schools using a proportionally stratified sample at multiple levels. First, a subdivision is made on the 
basis of the three levels of formal education: (1) primary education with pupils between 5 and 12 years old, 
(2) secondary education (12-18 years old) and (3) basic education (formal education for adults). Secondly, in 
primary and secondary education, a subdivision is made between general and special needs education. The 
total number of schools per group is examined and one-fifth of that number is chosen as the intended sample. 
The next stratum is based on (1) school size (small, medium, large), (2) educational network (three types) and 
province (five throughout Flanders and Brussels Capital Region). 

For the field study a final distinction is made between the three main actors in schools: pupils, teachers 
and principals. Since MICTIVO focuses on schools as a whole, it is important to include these different 
actors in the research. One fifth of the population is a considerably large sample and compliance of schools to 
take part in the monitor study is a challenge for researchers. In order to obtain the widest possible 
participation from the sampled schools, they are considered as subset units. In other words, MICTIVO tries 
to include a representative number of schools, represented by one or more of the three different actors. 
A school participates in the research if teachers and/or the principal and/or the pupils of that school 
participates. Given the time and resources, a funnel model was used to accommodate as many schools as 
possible (Goeman, Elen, Pynoo, van Braak, 2013). The funnel model consists of three sub studies. The first 
sub study contains 80% of all sampled schools for which only the principal is asked to participate. Sub study 
two consists of 10% of the schools where the principal and all the teachers are asked to participate. The last 
sub study also consists of 10% of the schools in the sample and in these schools the principal, the teachers 
and a number of classes of pupils were asked to participate. In primary education only the pupils of the fifth 
and sixth year (10-12 years old) were asked to participate. In secondary education a number of classes, 
dependent on the size of the school and the number of specializations of that school were asked to participate. 
As many different disciplines and grades were included to guarantee maximum representativity. 


3. DISCUSSION AND CONCLUSION 


The integration of ICT in education has been both a challenge and an interesting subject of study for 
governments, scholars and educational systems throughout the world. Countries are stimulating and 
monitoring ICT integration in various ways. The Flemish department of Education and Training wishes to 
have insight into the 'return on investment’, the effects of ICT on educational practices and emerging trends 
regarding ICT in education. As a consequence, the MICTIVO model and related measurement instruments 
were developed and empirically tested. 

The MICTIVO model has strong psychometric properties because of the repeated validation of the scales, 
the representative sample design, the large scale of the study and the validated comparison over time. This 
allows us to gather trustworthy information about different components of ICT integration using indicators 
that stand the test of time. The validated scales and the sampling design allows us to make reliable and 
representative claims about ICT integration in education. The size and breadth of the monitor ensures data on 
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which solid, independent advice to policy makers can be based. Furthermore, as the monitor consists of the 
same basic components throughout the different field studies since 2007-2008, one can have a closer look at 
emerging trends. These strong psychometric properties distinguish MICTIVO with the other monitors for 
ICT integration in Europe mentioned above on both representativity and the possibility for measuring 
changes throughout the years. 

Nevertheless, we wish to conclude this paper with a critical reflection. Setting up this particular research 
project and reviewing thoroughly other ICT monitoring projects made us consider seriously new questions. Is 
taking a snapshot of how and to what extent ICT is integrated in education still a priority when digitization 
seems to be deeply anchored in education? Should we not shift our focus towards potential undesirable 
effects and negative impacts of digitization on teaching and learning? And, evenly important: how is ICT 
going to be used in the future of education? 
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